Many types of computerized systems and computing devices, especially servers, are designed to handle requests for services, process these requests, and provide responses to the requests. The request may originate from other types of computing devices, especially clients, or from within the computing device handling the request itself. A service may be considered functionality provided by given software, where the service is interacted with by providing requests for the service to perform functionality, and in response by receiving responses to the requests in which the results of this functionality is provided.
For efficient utilization of computing resources, among other reasons, a processor that handles the requests for services desirably runs at full utilization, or 100%, instead of idling at some periods such that its utilization is less than 100%. However, running a processor at full utilization while providing timely responses to requests for services can be difficult. In particular, requests for services should be responded to within a given amount of time. Where this given amount of time is specified, therefore, it is desirable to minimize variations within the waiting times for responding to requests for services. If variations are not minimized, then performance can suffer, especially when the waiting times unduly increase.
For these and other reasons, therefore, there is a need for the present invention.